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DETAILED ACTION 



Double Patenting 

1. The nonstatutory double patenting rejection is based on a judicially created doctrine 
grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or 
improper timewise extension of the "right to exclude" granted by a patent and to prevent possible 
harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection 
is appropriate where the conflicting claims are not identical, but at least one examined 
application claim is not patentably distinct from the reference claim(s) because the examined 
application claim is either anticipated by, or would have been obvious over, the reference 
claim(s). See, e.g.. In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re 
Goodman, 1 1 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re LongU 759 F.2d 887, 225 
USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re 
Vogeh 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 
USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may 
be used to overcome an actual or provisional rejection based on a nonstatutory double patenting 
ground provided the conflicting application or patent either is shown to be commonly owned 
with this application, or claims an invention made as a result of activities undertaken within the 
scope of a joint research agreement. 

Effective January 1, 1994, a registered attorney or agent of record may sign a terminal 
disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 
3.73(b). 

2. Claims 1,2, 10, and 16 are provisionally rejected on the ground of nonstatutory 
obviousness-type double patenting as being unpatentable over claims 1,11,14, and 16 of 
copending Application No. 10/388,925. Although the conflicting claims are not identical, they 
are not patentably distinct from each other for the following reasons: 

Claim 1 of the present application discloses a method for controlling an object. Claim 1 
of Application No. 10/388,925 discloses the same method, but for specifically controlling a door. 
It would be extremely obvious to modify the method of controlling a door to control the motion 
of any object, such as a conveyor belt, for the purpose of turning on the conveyor belt when an 
object is present on its surface. 
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Claim 2 of the present application further limits claim 1 with a method for computing 3D 
features that is identical to claim 1 1 of Application No. 10/388,925. 

Claim 10 of the present application further limits claim 1 with a method for filtering that 
is identical to claim 16 of Application No. 10/388,925. 

Claim 16 provides determining heights of points relative to a ground plane and clustering 
the points in 3D space to generate objects, which is equivalent to claim 14 of AppUcation No. 
10/388.925. 

This is a provisional obviousness-type double patenting rejection because the conflicting 
claims have not in fact been patented. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed publication in this 
or a foreign country, before the invention thereof by the applicant for a patent. 

4. Claim 1 is rejected under 35 U.S.C. 102(a) as being anticipated by Schatz et al. (US 
Patent No 6,297,844), hereinafter referenced as Schatz. 

Regarding claim 1, Schatz discloses a video safety curtain comprising capturing a stereo 
image of an area to be monitored, which reads on "acquiring a stereo image of said viewed space 
wherein said stereo image comprises an image set", generating 3D features from the stereo 
image, which reads on "computing a set of 3D features from said stereo image", filtering the 3D 
features by comparing the 3D features to reference data, which reads on "filtering from said set 
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of 3D features to generate a set of filtered 3D features", calculating trajectories of the filtered 3D 
features, which reads on "computing a trajectory of said set of filtered 3D features", and 
generating a control signal that triggers an alarm based on the trajectory, wherein the alarm could 
control an objects motion, which reads on "generating a control signal influencing said objects 
motion in response to said trajectory", as disclosed at column 8 lines 28-52 and column 10 lines 
25-39. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made, 

6. Claims 14 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over Schatz. 
Regarding claim 14, Schatz discloses everything as applied above (see claim 1). Schatz 

fails to specifically disclose wherein calculating a trajectory comprises the step of correlating 
segmented features in a first frame with features around an expected object position in a 
following frame. However, the examiner take OFFICIAL NOTICE (see MPEP 2144.03) that it 
was extremely well known in the art to provide for wherein calculating a trajectory comprises the 
step of correlating segmented features in a first fi-ame with features around an expected object 
position in a following frame, and would therefore be obvious to modify Schatz by providing this 
method for trajectory calculation, for the purpose of increasing search speed when tracking an 
object firom frame to frame. 
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Regarding claim 16, Schatz discloses everything as applied above in claim 1. Schatz 
further discloses clustering 3D features into cluster-clouds in order to segment distinct 3D 
objects, which reads on "clustering said features having a height above said ground plane in 3D 
space to generate objects", since it is obvious that clusters of features acquired in 3D space, as 
taught by Schatz, have a height above the ground plane, and further discloses wherein calculating 
a trajectory of an object further comprises tracking its motion from frame to frame, which reads 
on "tracking said objects in multiple frames", as disclosed at column 9 lines 12-38 and column 
10 lines 25-40. Schatz does not specifically disclose measuring a height of the 3D features 
relative to the ground plane, however, the examiner takes OFFICIAL NOTICE (see MPEP 
2144.03) that it was extremely well known in the art to calculate the height of 3D features which 
is suggested by Schatz because the 3D features comprise position data, which reads on 
"measuring a height of said feature relative to a ground plane", and it would therefore be obvious 
to modify Schatz by providing for measuring a height of the 3D features relative to the ground 
plane for the purpose of filtering out 3D features based on their height. 

7. Claims 2, 3, 9, 12, and 17 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Schatz in view of Dhond et al. ("Structure from Stereo - A Review"), hereinafter referenced as 
Dhond. 

Regarding claim 2, Schatz discloses everything as applied above (see claim 1). Schatz 
further discloses where the 3D features are derived by a well-known edge segmentation process, 
as disclosed at column 7 lines 38-44. However, Schatz fails to specifically disclose the details of 
computing the 3D features. However, the examiner maintains that it was well known in the art 
to provide calculating 3D features by edge-processing a stereo image to generate a plurality of 
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connected edges, identifying connected edges having length greater than a predetermined 
threshold as features, matching features to generate disparities generated from different images 
in an image set, and computing 3D locations of feature points according to the disparities and 
camera geometry, as taught by Dhond. 

In the same field of endeavor, Dhond discloses 3D feature calculating methods 
comprising edge-processing a stereo image to generate a plurality of chained together edges, 
wherein it is well known to filter out edges based on length for the purpose of decreasing noise, 
as disclosed at page 1490, and further discloses matching features to generate disparities 
* generated firom different images in said image set, and computing 3D locations of feature points 
according to said disparities and camera geometry, as disclosed at page 1492 column 1, which 
reads on "edge-processing said stereo image to generate a plurality of cormected edgelets; 
identifying connected edgelets having length greater than a predetermined threshold as features; 
matching features to generate disparities generated fi-om different images in said image set; and 
computing 3D locations of feature points according to said disparities and camera geometry". 

Therefore it would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Schatz by providing identifying connected edges, from the set of edges (3D 
features) acquired by Schatz, having length greater than a predetermined threshold as features, 
matching features to generate disparities generated from different images in said image set, and 
computing 3D locations of feature points according to said disparities and camera geometry, as 
taught by Dhond, for the purpose of accurately locating 3D objects using a stereo camera. 

Regarding claim 3, Schatz and Dhond disclose everything as applied above (see claim 2). 
Dhond further discloses applying stereo techniques to trinocular stereo comprising generating 
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depth maps and therefore disparities, since depth is calculated from disparity, from a horizontal 
depth map and a vertical depth map, which reads on "merging horizontal and vertical disparities 
to form a set of selected disparities'', as disclosed at page 1503. 

Regarding claim 9, Schatz and Dhond disclose everything as applied above (see claim 3). 
Schatz and Dhond fail to specifically give orientation ranges for selecting a horizontal disparity 
or vertical disparity of a feature. However, the examiner takes OFFICIAL NOTICE (see MPEP 
2144.03) that it was extremely well known in the art to provide orientation ranges for selecting a 
horizontal disparity or vertical disparity when merging two disparity images in Trinocular 
Stereo, wherein it would be obvious to use the ranges as claimed, since the applicant does not 
disclose any distinct benefit of the particular ranges as claimed. Therefore it would have been 
obvious to modify Schatz and Dhond by providing orientation ranges for selecting a horizontal 
disparity or vertical disparity when merging two disparity images in Trinocular Stereo, wherein 
horizontal disparity is selected if the orientation of the feature is between 45 and 135 degrees or 
225 and 315 degrees and wherein vertical disparity is selected otherwise, which reads on 
"merging step includes the steps of multiplexing said disparities by: selecting said horizontal 
disparities to be passed along if an orientation of said feature is between 45 and 135 or between 
225 and 315; and selecting said vertical disparities to be passed along if said orientation of said 
feature is not between 45 and 135 or between 225 and 315", for the purpose of improving object 
depth measurements by using trinocular stereo. 

Regarding claim 12, it is interpreted and thus rejected for the same reasons as applied 
above in the rejection of claim 2, because in order to match features to generate disparities as 
disclosed in claim 2, a rectification process must be performed between a left image and a right 
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image, wherein then the resulting disparity image is obtained and the features are found within 
the disparity image thus giving sparsified (features) disparities. 

Regarding claim 17, Schatz discloses everything as applied above in the rejection of 
claim 1 . Schatz further discloses capturing a plurality of 3D images including a reference image, 
and tracking an object in a plurality of images, thereby creating a trajectory, wherein the scene is 
monitored using the trajectory, as disclosed at column 6 line 39 - column 7 line 3 and column 10 
lines 25-40. However, Schatz fails to specifically disclose a disparity map, however since Schatz 
discloses tracking an object wherein the object is specified by a location including a depth value, 
it would be obvious for Schatz to use a disparity map in order to calculate a 3D depth value for 
an object, as taught by Dhond. 

In the same field of endeavor, Dhond discloses where the disparity obtained by 
computing the relative displacement of the matching feature points in the two images in a stereo 
image is used to extract the 3D depth of the point. 

Therefore, it would have been obvious to one of ordinary skill in the art at the time of 
invention to modify Schatz by using a disparity map for the purpose of consolidating disparity 
values used in the object depth calculation. 

8. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Schatz in view of 
Dhond further in view of Admitted Prior Art (page 9 lines 21-25 of the specification), hereinafter 
referenced as Admitted. 

Regarding claim 5, Schatz and Dhond disclose everything as applied above (see claim 2). 
However, Schatz and Dhond fail to specifically disclose detecting features by performing a 
parabolic smoothing step, a non-integral sub-sampling step at a predefined granularity, a sobel 
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edge detection step, a true peak detection step, and a chaining step. However, the examiner 
maintains that it was well known in the art to provide detecting features by performing a 
parabolic smoothing step, a non-integral sub-sampling step at a predefined granularity, a sobel 
edge detection step, a true peak detection step, and a chaining step, as taught by Admitted at page 
9 lines 21-25. 

Therefore, it would have been obvious to one of ordinary skill in the art to modify Schatz 
by providing detecting features by performing a parabolic smoothing step, a non-integral sub- 
sampling step at a predefined granularity, a sobel edge detection step, a true peak detection step, 
and a chaining step, as taught by Admitted, for the purpose of improving feature detection by 
finding edges more accurately. 

9. Claims 6, 7, and 8 are rejected under 35 U.S.C. 103(a) as being unpatentable over Schatz 
in view of Dhond fiirther in view of Pollard et al. ("PMF: A Stereo Correspondence Algorithm 
Using a Disparity Gradient Limit"), hereinafter referenced as Pollard. 

Regarding claim 6, Schatz and Dhond disclose everything as applied above (see claim 2). 
Dhond discloses as discussed above, matching features from a first image to a second image to 
identify disparities. Dhond further discloses constraining an initial set of possible matches of 
said disparities for each feature using an epipolar constraint, as disclosed at page 1491 section B. 
However, Schatz and Dhond fail to specifically disclose characterizing each of the possible 
matches by an initial strength of match, by comparing the strength and orientation of the edges, 
and enforcing a smoothness constraint within a pre-selected allowable disparity gradient. 
However, the examiner maintains that it was well known in the art to provide for characterizing 
each of the possible matches by an initial strength of match, by comparing the strength and 
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orientation of the edges, and enforcing a smoothness constraint within a pre-selected allowable 
disparity gradient, as taught by Pollard. 

In the same field of endeavor, Pollard discloses a PMF stereo correspondence algorithm 
comprising applying an epipolar constraint, characterizing each of the possible matches, between 
a left and a right image, by an initial strength match strength by comparing edges, and enforcing 
a smoothness constraint within a pre-selected allowable disparity gradient, as disclosed at page 
453-455, which reads on "characterizing each of said possible matches by an initial strength of 
match (SOM), by comparing the strength and orientation of said edgelets; and enforcing a 
smoothness constraint within a pre-selected allowable disparity gradient". 

Therefore it would have been obvious to one of ordinary skill in the art to modify Schatz 
and Dhond by providing characterizing each of the possible matches by an initial strength of 
match, by comparing the strength and orientation of the edges, and enforcing a smoothness 
constraint within a pre-selected allowable disparity gradient as taught by Pollard for the purpose 
of solving the stereo correspondence problem, improving the accuracy of the matching, and 
reducing computational cost by using a simple algorithm. 

Regarding claim 7, Schatz, Dhond, and Pollard disclose everything as applied above (see 
claim 6). Pollard further discloses updating the match strength of each correspondence, 
comparing correspondences neighboring features under consideration, and enforcing uniqueness 
by iteratively identifying matches having a maximum matching strength for both of its 
constituent features and eliminating all other matches associated with each constituent of the 
identified match, as disclosed at page 454-455. 
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Regarding claim 8, Schatz, Dhond, and Pollard disclose everything as applied above (see 
claim 6). Dhond further discloses wherein features from the right and left images are merged to 
identify horizontal depth and therefore disparities, and further matching features from the left 
image to a top image to identify vertical depth and therefore disparities, as disclosed at page 
1503 column 2, which reads on "wherein features from said right and left images are merged to 
identify horizontal disparities; and further matching features from a either said right or left image 
to a top image to identify vertical disparities". 

10. Claims 4, 1 1, and 15 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Schatz in view of Burschka at al. ("Scene Classification from Dense Disparity Maps in Indoor 
Environments"), hereinafter referenced as Burschka. 

Regarding claim 4, Schatz discloses everything as applied above (see claim 1). As 
discussed above in the rejection of claim 16, Schatz discloses clustering 3D features into cluster- 
clouds in order to segment distinct 3D objects. However, Schatz fails to specifically disclose 
filtering ground plane noise from the objects. However, the examiner maintains that it was well 
known in the art to filter ground plane noise from an object, as taught by Burschka. 

In the same field of endeavor, Burschka discloses a method for generating a 3D model of 
a scene using a stereo camera system comprising segmenting foreground objects from the 
background, wherein the background comprises a ground plane, by removing disparities 
corresponding to the ground plane from the disparity map, thereby segmenting out noise caused 
by shadows or objects on the ground, as disclosed at section 3 and section 4.1, which reads on 
"wherein said set of filtered 3D features are generated by filtering ground plane noise from said 
objects". 
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Therefore, it would have been obvious to one of ordinary skill in the art to modify Schatz 
by filtering ground plane noise from the objects, as taught by Burschka, for the purpose of not 
detecting shadows as moving objects. 

Regarding claim 11, Schatz and Burschka disclose everything as applied above (see 
claim 4). Although Schatz and Burschka each disclose a segmenting step for segmenting objects 
in a 3D image, they fail to specifically disclose selecting objects wherein a 2D distance between 
the objects exceeds a preset threshold. However, the examiner takes OFFICIAL NOTICE (see 
MPEP 2144.03) that it was extremely well known in the art to select objects wherein a 2D 
distance between the objects exceeds a preset threshold. Therefore, it would have been obvious 
to modify the segmenting step of Schatz and Burschka by selecting objects wherein a 2D 
distance between the objects exceeds a preset threshold, for the purpose of locating objects that 
are truly distinct. 

Regarding claim 15, it is interpreted and thus rejected for the same reasons as applied 
above in the rejection of claims 1 and 4. 

1 1 . Claim 1 0 is rejected under 35 U.S.C. 103(a) as being unpatentable over Schatz in view of 
Burschka further in view of Weng ("Agglomerative Clustering Algorithm"). 

Regarding claim 10, Schatz and Burschka disclose everything as applied above (see 
claim 4). However, they fail to specifically disclose breaking chains of features into contiguous 
segments based on abrupt changes in z between successive points. However, the examiner takes 
OFFICIAL NOTICE (see MPEP 2144.03) that it was extremely well known in the art to break 
edges into separate objects based on depth values. Therefore it would have been obvious to 
modify Schatz and Burschka by providing breaking chains of features into contiguous segments 
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based on abrupt changes in z between successive points, for the purpose of separating objects 
that are located at different depths within a scene, since edges are initially located in only two 
dimensions. 

However, Schatz and Burschka further fail to disclose merging the two closest clusters 
based on a minimum distance criterion. However, the examiner maintains that it was well 
known in the art to merge two closest clusters based on a minimum distance criterion, as taught 
by Weng. 

In the same field of endeavor Weng discloses an agglomerative clustering algorithm 
comprising choosing two clusters with the smallest distance and replacing these two clusters 
with a new cluster formed by merging the original two clusters, which reads on "merging two 
closest clusters based on a minimum distance criteria". 

Therefore it would have been obvious to one of ordinary skill in the art to modify Schatz 
and Burschka, by providing for merging the two closest clusters based on a minimum distance 
criterion, for the purpose of reducing noise in the clustering step, 

12. Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over Schatz in view of 
Michael et al. (US Patent No 6,173,070), hereinafter referenced as Michael. 

Regarding claim 13, Schatz discloses everything as applied above (see claim 1). 
However, Schatz fails to specifically disclose converting 3D features to a ground plane 
coordinate system, eliminating features having insufficient distance from the ground plane, 
projecting remaining features onto the ground plane, converting the projected features to a 2D 
image, generating distinct regions, scoring features in the distinct regions, and comparing the 
region scores to a threshold to determine if an object is present or absent. However, the 
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examiner maintains that it was well known in the art to provide converting 3D features to a 
ground plane coordinate system, eliminating features having insufficient distance from the 
ground plane, projecting remaining features onto the ground plane, converting the projected 
features to a 2D image, generating distinct regions, scoring features in the distinct regions, and 
comparing the region scores to a threshold to determine if an object is present or absent, as 
taught by Michael. 

In the same field of endeavor, Michael discloses a method for finding features in a 3D 
stereo image comprising estimating a ground plane in a scene, calculating height of a feature 
relative to the ground plane, eliminating features less than a threshold away firom the ground 
plane, projecting features onto the ground plane, thereby acquiring a 2D image, breaking the 
scene into a plurality of windows and determining for each window if features in the window are 
ball data points or non-ball data points based on a score calculated from the features in the 
window, as disclosed at column 9 lines 19-50, which reads on "converting said 3D features to a 
ground plane coordinate system; eliminating features having excessive or insufficient range, 
excessive lateral distance, excessive height, or insufficient distance from said ground plane; 
projecting remaining features into said ground plane to generate projected features; converting 
said projected features to a 2D image; obtaining distinct regions wherein each pixel represents a 
plurality of feature points; scoring features in said distinct region using a scoring fiinction to 
generate region scores; accumulating said region scores and comparing said accumulated scores 
to a predetermined threshold to determine if an object is present or absent". 

Therefore it would have been obvious to one of ordinary skill in the art to modify Schatz 
by providing converting 3D features to a ground plane coordinate system, eliminating features 
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having insufficient distance from the ground plane, projecting remaining features onto the 
ground plane, converting the projected features to a 2D image, generating distinct regions, 
scoring features in the distinct regions, comparing the region scores to a threshold to determine if 
an object is present or absent, as taught by Michael, for the purpose of filtering out features that 
are close to the ground plane in order to detect only features that are important to the detection 
system, thereby decreasing tracking computations. 



Conclusion 

13. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. Nichani (US Patent No 6,701,005) discloses a method for 3D object segmentation by 
one of the inventors of the present application. WoodfiU et al. (US Patent No 6,215,898) 
disclose an image processing method for stereo correspondence. Hattori et al. (US Patent No 
6,963,661) disclose an obstacle detection system comprising capturing a stereo image of a scene 
and using it for object detection, further comprising computing the height of each object relative 
to a ground plane, and eliminating consideration of objects less than a certain height. Bramblet 
et al. (US Pub No 2004/0017929) disclose a method for detecting reverse entry though a door, 
comprising acquiring a stereo image of a door area, processing the image in order to locate and 
track objects, further comprising controlling a door in response to an objects motion and 
trajectory. Jain et al. (Machine Vision - Chapter 1 1) disclose a method for stereo imaging 
comprising matching features from the two images in a stereo image in order to generate 
disparities and computing 3D locations of points based on the disparities and camera geometry, 
wherein depth of a point in a scene is specifically calculated from the disparity between that 
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point in the two images, the distance between the two camera lens centers, and the focal length. 
Harville (US Patent No. 7,003,136) discloses a method for object tracking comprising generating 
a 3D point cloud of an object and mapping the point cloud to one or more plan-view images, 
wherein each plan-view image contains a plurality of vertical oriented bins. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jordan Kuhn whose telephone number is 571-272-4295. The 
examiner can normally be reached on M-F 8:00AM-4:30PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph Mancuso can be reached on 571-272-7695. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
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